Noise reduction apparatus, method and program

ABSTRACT

Uncorrelated noise is reduced in image data. To achieve this, CCD-RAW data, in which color components are output in order in accordance with an array of color filters that have been formed on the photoreceptor surface of a CCD, is subjected to array conversion processing in such a manner that the data is divided into its color components. Image data indicating pixels within a noise-reduction target area of 5×5 pixels is extracted from the CCD-RAW data obtained by the array conversion. A filter for reducing uncorrelated noise is calculated and a filtering operation is performed using the calculated filter while the correlativity of the CCD-RAW data is maintained. These processing steps are repeated with regard one frame of the CCD-RAW data.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a noise reduction apparatus and method and toa noise reduction program.

2. Description of the Related Art

CCDs used in digital still cameras are continuing to be improved interms of number of pixels and sensitivity. The influence of noise,therefore, has become a problem.

Use of a low-pass filter or median filter, etc., to remove noise from avideo signal obtained by sensing an image has been considered (see thespecification of Japanese Patent Application Laid-Open No. 4-235472).Further, the removal of noise from an image without detracting fromimage sharpness also has been considered (see the specification ofJapanese Patent Application Laid-Open No. 2002-222416).

It is still difficult, however, to remove noise completely.

SUMMARY OF THE INVENTION

Accordingly, an object of the present invention is to reduce noise.

According to the present invention, the foregoing object is attained byproviding a noise reduction apparatus comprising: a color-componentextraction device (color-component extraction means) for inputting imagedata representing an image in which each pixel of a number of pixelsconstituting one frame of an image has one color component from among aplurality of color components, the color components having adistribution that is systematic, and extracting, on aper-color-component basis, color image data representing a pixel withina zone in which the color image data is regarded as havingcorrelativity; a noise removal device (noise removal means) for removinguncorrelated noise, which is contained in the color image data that hasbeen extracted on a per-color-component basis, while maintainingcorrelativity of the color image data that has been extracted by thecolor-component extraction device on a per-color-component basis; and acontrol device for exercising control in such a manner thatcolor-component extraction processing by the color-component extractiondevice and noise removal processing by the noise removal device isrepeated with regard to the one frame of color image data.

The present invention also provides a method suited to the noisereduction apparatus described above. Specifically, the present inventionprovides a noise reduction method comprising the steps of: inputtingimage data representing an image in which each pixel of a number ofpixels constituting one frame of an image has one color component fromamong a plurality of color components, the color components having adistribution that is systematic, and extracting, on aper-color-component basis, color image data representing a pixel withina zone regarded as having correlativity; removing uncorrelated noise,which is contained in the color image data that has been extracted on aper-color-component basis, while maintaining correlativity of the colorimage data that has been extracted on a per-color-component basis; andrepeating the color-component extraction processing and noise removalprocessing with regard to the one frame of color image data.

The present invention also provides a program for implementing the noisereduction processing described above.

In accordance with the present invention, uncorrelated noise is removedfrom image data representing an image in which each pixel of a number ofpixels constituting one frame of an image has one color component fromamong a plurality of color components, the color components having asystematic distribution. Since such image data is data that prevailsprior to application of interpolation processing (processing forgenerating a color component that does not exist among the plurality ofcolor components), it is possible to implement noise reductionprocessing without dependence upon circuit structure, such as a signalprocessing circuit used after interpolation processing, in comparisonwith a case where noise removal processing is executed afterinterpolation processing.

The apparatus may further comprise a noise removal processing inhibitingdevice (noise removal processing inhibiting means) for inhibiting noiseremoval processing by the noise removal device in accordance with thelevel of the color image data.

The color-component extraction device inputs color image data that isoutput from a single-chip solid-state electronic image sensing device inwhich a number of optoelectronic transducers are arrayed, color filterseach having a characteristic that passes light of one color componentfrom among the plurality of color components being formedsystematically, for each of the plurality of colors, on photoreceptorsurfaces of respective ones of the optoelectronic transducers of thenumber of optoelectronic transducers.

It may be so arranged that processing for removing uncorrelated noise inthe noise removal device is executed based upon at least one of thecharacteristic of the solid-state electronic image sensing device andshooting information used when a picture has been taken using thesolid-state electronic image sensing device.

The noise removal device includes a color image data shifting device(color image data shifting means) for shifting the level of the colorimage data, which has been extracted by the color-component extractiondevice, in such a manner that an average value of the levels of thecolor image data that has been extracted by the color-componentextraction device will become an origin position of color space of theplurality of color components; a filtering device (filtering means) forsubjecting the color image data that has been shifted by the color imagedata shifting device to processing for removing uncorrelated noise inaccordance with the level of this color image data; and a color imagedata reverse shifting device for returning, in accordance with theamount of shift, the level of the color image data from which theuncorrelated noise has been removed by the filtering device.

Thus, uncorrelated noise can be removed while the correlativity of thecolor image data is maintained.

The noise reduction processing in the filtering device may be digitalfiltering processing that utilizes calculation conforming to the numberof the plurality of color components.

Further, the noise reduction processing in the filtering device may bedigital filtering processing for handling the plurality of colorcomponents in such manner that the number of the plurality of colorcomponents will become less than the number of original colorcomponents, this processing utilizing calculation conforming to thenumber of the plurality of color components. The amount of calculationinvolved in digital filtering processing can be reduced.

The apparatus may further comprise a noise reduction processing haltingdevice (noise reduction processing halting means) for halting noisereduction processing in a case where calculation in the digitalfiltering processing has diverged.

Digital filtering processing applied to the shifted color image data inthe filtering device uses processing that subtracts uncorrelated noisefrom image data in an area that contains uncorrelated noise, by way ofexample.

The apparatus may further comprise an image data dividing device (imagedata dividing means) for dividing the entirety of the color image datathat has been extracted by the color-component extraction device into aplurality of color image data groups exhibiting correlativity with oneanother. In this case, the color image data shifting device wouldperform the shift with regard to a color image data group, whichcontains color image data that is to undergo removal of uncorrelatednoise, from among the plurality of color image data groups obtained bydivision by the image data dividing device, in such a manner that thiscolor image data group will occupy the origin position of the colorspace of the plurality of color components. An image represented bycolor image data that has undergone noise reduction processing can beprevented from being degraded owing to use of image data that has nocorrelativity.

The image data dividing device includes a small-block dividing device(small-block dividing means) for dividing color image data that has beenextracted by the color-component extraction device into a plurality ofsmall blocks. In this case, the color image data is divided into aplurality of correlated color image data groups per each of theplurality of small blocks obtained by division by the small-blockdividing device.

It may be so arranged that the color image data is divided into aplurality of correlated color image data groups per each of theplurality of small blocks based upon representative pixels thatconstitute the small blocks.

The apparatus may further comprise an uncorrelated noise halting controldevice (uncorrelated noise halting control means) for halting processingfor removing uncorrelated noise by the uncorrelated noise removingdevice in a case where the size of an image represented by the pluralityof color image data groups obtained by division by the image datadividing device is less than a first predetermined value.

In a case where the size of an image area represented by the pluralityof color image data groups is equal to or greater than a secondpredetermined value larger the first predetermined value, the shift maybe performed with regard to a color image data group, which containscolor image data that is to undergo removal of uncorrelated noise, fromamong the plurality of color image data groups obtained by division bythe image data dividing device.

The apparatus may further comprise a luminance data generating device(luminance data generating means) for generating luminance data from thecolor image data that has been extracted by the color-componentextraction device. In this case, the color image data dividing devicewould divide the color image data into a plurality of luminance datagroups, which exhibit correlativity with one another, from among theluminance data that has been generated by the luminance data generatingdevice. The amount of calculation can be reduced in this case as well.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates part of the photoreceptor surface of a CCD;

FIG. 2 illustrates arrays of pixels after division according to color;

FIGS. 3 to 6 illustrate the relationship between color space and imagedata;

FIG. 7 is a block diagram illustrating the electrical structure of adigital still camera;

FIG. 8 illustrates the relationship between standard deviation of noiseand pixel values;

FIG. 9 is a block diagram illustrating the electrical structure of anoise reduction circuit;

FIG. 10 is a block diagram illustrating the electrical structure of acomputer system;

FIG. 11 is a flowchart illustrating noise reduction processing;

FIG. 12 illustrates an example of the image of a subject;

FIG. 13 illustrates the relationship between color space and image data;

FIGS. 14 and 15 illustrate examples of areas to undergo noise reduction;

FIG. 16 is a flowchart illustrating processing for area discriminationand the like;

FIGS. 17 and 18 illustrate examples of areas to undergo noise reduction;

FIGS. 19 and 20 illustrate part of the photoreceptor surface of a CCD.

DESCRIPTION OF THE PREFERRED EMBODIMENT

A preferred embodiment of the present invention will now be described indetail with reference to the drawings.

The principles of noise reduction processing according to a preferredembodiment of the invention will be described first.

FIG. 1 illustrates part of the photoreceptor surface of a CCD 1 used ina digital still camera according to this embodiment.

Here 4096 photodiodes 2 are disposed in the column direction of the CCD1 and 1540 photodiodes 2 in the row direction. Accordingly, image datarepresenting the image of a subject composed of 4096 pixels in thecolumn direction and 1540 pixels in the row direction is obtained bysensing the image of the subject using the CCD 1. The image dataobtained from each photodiode 2 corresponds individually to each pixelof the image.

Formed on the photoreceptor surface of each photodiode 2 of themultiplicity thereof is a color filter having a characteristic thatpasses any one color component among a plurality of color components,namely a red color component, blue color component, first green colorcomponent and second green color component (the first and second greencolor components may have identical characteristics). The filters thatpass the red color component, blue color component, first green colorcomponent and second green color component have been assigned thecharacters “R”, “B”, “G1” and “G2”, respectively.

The filters that pass the red light component and the filters that passthe blue light component are formed alternately on the photoreceptorsurfaces of the photodiodes 2 in odd-numbered rows. The filters thatpass the first green light component and the filters that pass thesecond green light component are formed alternately on the photoreceptorsurfaces of the photodiodes 2 in even-numbered rows.

In this embodiment, noise reduction processing is executed using an area(a noise-reduction target area A), which is composed of ten photodiodes2 in each of the column and row directions, as a single unit. Althoughthe physical positions (spatial positions) of four photodiodes 2 thatadjoin one another in the column and row directions are different, thesephysical positions are regarded as being the same in this embodiment.Four pixels that correspond to four photodiodes 2 are handled as one set(though it goes without saying that the number of pixels handled as aset need not necessarily be four). The noise-reduction target area Acontains 25 of these sets (X1 to X25) of photodiodes 2.

When the image of the subject is sensed using such a CCD 1, the CCD 1outputs CCD-RAW data representing the image of the subject. The CCD-RAWdata is such that items of data having color components that are inaccordance with the color filters that have been formed on thephotoreceptor surfaces of the photodiodes 2 appear serially one row'sworth at a time. A pixel is represented by data of any one colorcomponent, namely the red color component, blue color component, firstgreen color component or second green color component; it does notpossess data of another color component. Data of another color componentis interpolated by color interpolation processing.

FIG. 2 illustrates how an image looks after execution of color-componentdivision processing.

In this embodiment, color-component extraction processing(color-component division processing) of CCD-RAW data is executed insuch a manner that an array of pixels (this pixel array corresponds tothe array of photodiodes 2 in FIG. 1) of an image represented by theCCD-RAW data that has been output from the CCD 1 in the manner describedabove will represent an image that has been divided on aper-color-component basis.

The image represented by this image data after it has been subjected tocolor-component division processing has 4096 pixels in the columndirection and 1540 pixels in the row direction, as described above. Theimage after division into color components can be divided into anupper-left area, lower-left area, upper-right area and lower-right areaon a per-color-component basis. All of these areas have 2048 pixels inthe column direction and 770 pixels in the row direction.

The upper-left area, lower-left area, upper-right area and lower-rightarea are image portions IR, IG, IG1 and IG2 represented by the imagedata of the red color component, blue color component, first green colorcomponent and second green color component, respectively.

Areas of five pixels in each of the column and row directions in each ofthese image portions IR, IB, IG1 and IG2 become noise-reduction targetareas AR, AB, AG1 and AG2, respectively. The noise-reduction targetareas AR, AB, AG1 and AG2 in combination correspond to thenoise-reduction target area shown in FIG. 1, as mentioned above.

Pixels Rc, Bc, G1c and G2c at the centers of the noise-reduction targetareas AR, AB, AG1 and AG2, respectively, are pixels that are to undergonoise reduction. As will be described later in greater detail, noisereduction processing of the central pixels Rc, Bc, G1c and G2c isexecuted utilizing the pixels (image data) present in thenoise-reduction target areas AR, AB, AG1 and AG2. When noise reductionprocessing of the central pixels Rc, Bc, G1c and G2c of thenoise-reduction target areas AR, AB, AG1 and AG2, respectively, ends,the noise-reduction target areas AR, AB, AG1 and AG2 are each shiftedone pixel to the right and noise reduction processing is applied to thepixels Rc, Bc, G1c and G2c located at the centers of respective ones ofthe noise-reduction target areas AR, AB, AG1 and AG2 thus shifted.Shifting of the noise-reduction target areas and noise reductionprocessing are thus repeated for one frame of the image.

As mentioned above, the positions of four pixels adjoining one anotherin the column and row directions in the CCD 1 differ physically but thepixels are regarded as being at the same position in this embodiment.This means that a pixel within the noise-reduction target areas AR, AB,AG1 and AG2 can be expressed by Xn=(Rn, Gn, Bn, G2n). (Since the pixelswithin the noise-reduction target areas AR, AB, AG1 and AG2 are 25 innumber, n=1 to 25 holds.)

FIGS. 3 to 6 illustrate the relationship between color space of the red,blue, first green and second green color components and image datarepresenting a pixel in a noise-reduction target area.

FIG. 3 illustrates the relationship between the color space and imagedata representing a pixel in a noise-reduction target area A0 in a casewhere there is no uncorrelated noise.

The noise-reduction target area A0 is regarded as one havingcorrelativity. Image data representing a pixel Xn0=(Rn0, G1no, Bn0,G2n0) within the noise-reduction target area A0, therefore, falls withinbounds in which the levels of the image data are comparativelyconsolidated. Average data of pixel Xn0 within the noise-reductiontarget area A0 is indicated by XAV0.

FIG. 4 illustrates the relationship between the color space and imagedata representing a pixel in a noise-reduction target area A1 in a casewhere uncorrelated noise is present. The noise-reduction target area A0for the case where there is no uncorrelated noise also is illustratedfor the purpose of comparison.

In a case where uncorrelated noise is present, image data representing apixel Xn1=(Rn1, G1n1, Bn1, G2n1) within the noise-reduction target areaA1 is such that the levels of the image data representing each of thepixels are dispersed owing to the uncorrelated noise. Consequently, thezone of the image data representing pixels in the noise-reduction targetarea A1 in a case where uncorrelated noise is present is broader thanthe zone of image data representing pixels in the noise-reduction targetarea A0 in a case where uncorrelated noise is absent. Further, theaverage data of pixel Xn0 in the noise-reduction target area A1 isindicated by XAV1.

Noise reduction processing according to this embodiment eliminatesuncorrelated noise.

FIG. 5 illustrates the relationship between the color space and imagedata representing a pixel in the noise-reduction target area A1 at thetime of movement of image-data coordinates and filtering for removal ofnoise.

In a case where noise reduction is executed in this embodiment, all ofthe image data representing pixel Xn1 in noise-reduction target area A1undergoes a coordinate shift (level shift) in such a manner that thelevel of the average data XAV1 of pixel Xn1 in noise-reduction targetarea A1 will become the origin of the color space. Filtering serving asnoise reduction processing is applied to the image data representing thepixel Xn1 in the noise-reduction target area A1 in a state in which allof the image data representing pixel Xn1 in noise-reduction target areaA1 has undergone a coordinate shift in such a manner that the averagedata Xn1 is shifted to the origin position. Since filtering is appliedwith the average data XAV1 as the position of the origin, comparativelyappropriate filtering can be performed.

FIG. 6 illustrates the relationship between the color space and imagedata representing a pixel in the noise-reduction target area A1 at thetime of reverse movement of image-data coordinates.

The above-described filtering processing eliminates uncorrelated noise.Owing to this filtering processing, the zone of image data representinga noise-reduction target pixel within the noise-reduction target area A1falls within (approaches) the zone of image data representing a pixelwithin the noise-reduction target area A0 in a case where there is nouncorrelated noise. By repeating similar processing also with regard tothe remaining pixels in the noise-reduction target area A1, all of thepixels in the noise-reduction target area A1 fall within (approach) thezone of the image data representing pixels in the noise-reduction targetarea A0.

When the image data representing the pixel Xn1 in the noise-reductiontarget area A1 is subjected to filtering as noise reduction processingin a state in which all of the image data representing the pixel Xn1 inthe noise-reduction target area A1 has had its coordinates shifted, asmentioned above, a coordinate reverse-shift (a level reverse-shift) isperformed in such a manner that the average data XAV1 returns to theposition that prevailed prior to the shift of coordinates. Noisereduction processing is thus completed.

FIG. 7 is a block diagram illustrating the electrical structure of adigital still camera in which the above-described noise reductionprocessing is executed.

By sensing the image of a subject, a video signal is output from the CCD1 as described above and is input to an analog/digital convertingcircuit 3. The latter outputs the above-mentioned CCD-RAW data.

In a case where CCD-RAW recording has been set by a recording modeswitch (not shown), the CCD-RAW data that has been output from theanalog/digital converting circuit 3 is input to a recording controlcircuit 10. The CCD-RAW data is recorded on a memory card 11 by therecording control circuit 10.

In a case where recording of compressed data has been set by therecording mode switch, the CCD-RAW data that has been output from theanalog/digital converting circuit 3 is input to a color balanceadjustment circuit 4 and is subjected to a color balance adjustment. TheCCD-RAW data that has undergone the color balance adjustment issubjected to a gamma correction in a gamma correction circuit 5 and isthen input to a noise reduction circuit 6.

The noise reduction circuit 6 executes noise reduction based upon theabove-described principle. The noise reduction processing in noisereduction circuit 6 will be described later in greater detail.

The image data that has been output from the noise reduction circuit 6is input to a synchronization/YC generating circuit 8. The latterexecutes synchronization processing for generating image data of a colorcomponent that does not exist in the image data representing the imageand executes processing for generating luminance data and colordifference data. The generated luminance data and color difference datais compressed in a compression circuit 9. The compressed luminance dataand color difference data is recorded on the memory card 11 by therecording control circuit 10.

Further, a luminance data generating circuit 7 may be provided in orderto generate luminance data from the CCD-RAW data output from the gammacorrection circuit 5 and apply noise reduction processing using thegenerated luminance data. Noise reduction processing utilizing luminancedata will be described later in greater detail (see FIGS. 14 and 15).

The amount of noise used in noise reduction processing will be describedbefore the details of noise reduction processing.

FIG. 8 illustrates the relationship between amount of noise used innoise reduction processing and pixel values.

In noise reduction processing according to this embodiment, the image ofa prescribed reference subject is sensed and an amount Dn of noise isanalyzed in advance for every pixel level of each of a red colorcomponent, blue color component, first green color component and secondgreen color component.

As the pixel levels of the red color component, blue color component,first green color component and second green color component becomegreater, noise amounts D_(nR), D_(nB), D_(nG1) and D_(nG2) of thesecolor components, respectively, increase and peak at certain values. Ifthe pixel values increase further, then the noise amounts D_(nR),D_(nB), D_(nG1) and D_(nG2) gradually decrease.

This relationship between the noise amounts D_(nR), D_(nB), D_(nG1) andD_(nG2) and pixel levels of each of the color components is analyzedbeforehand and stored. The stored noise amounts D_(nR), D_(nB), D_(nG1)and D_(nG2) are utilized in noise reduction processing, described later.

Although the above-mentioned noise amounts D_(nR), D_(nB), D_(nG1) andD_(nG2) can also be utilized as is, gain WBG of the color balanceadjustment circuit 4 may be utilized. A noise amount D_(nG1)(γ) in acase where the gain WBG of the color balance adjustment circuit 4 isutilized is represented by the Equation (1) below.D _(nG1)(γ)=D _(nG1) ×[WBG] ^(γ)  Equation (1)

In Equation (1), the gain WBG of color balance adjustment circuit 4 ismultiplied by γ because CCD-RAW data following a γ conversion issubjected to noise reduction processing in this embodiment. It goeswithout saying that in a case where noise reduction processing isapplied to CCD-RAW data prior to the γ conversion, gain WBG notmultiplied by γ is utilized. The noise amount D_(nG2)(γ) of the secondgreen color component also is obtained in similar fashion. In a casewhere noise reduction processing is applied to CCD-RAW data before thecolor balance adjustment, it will suffice to use the noise amountD_(nG1) itself.

The noise amounts D_(nB)(γ) and D_(nR)(γ) of the blue and red colorcomponents are represented by Equations (2) and (3) below, in which[WBG] in Equation (1) has been replaced by [WBG(R/G)] and [WBG(B/G)],respectively.D _(nR)(γ)=D _(nR) ×[WBG(R/G)]^(γ)  Equation (2)D _(nB)(γ)=D _(nB)(γ)×[WBG(B/G)]^(γ)  Equation (3)

It goes without saying that amounts of noise can be calculated usingimaging information other than color balance. Examples of imaginginformation are the characteristics of the image sensing device such asCCD 1, a shading characteristic, ISO sensitivity, γ characteristic, SRcombining ratio, dynamic-range characteristic, automatic exposureamplifier for when a flash of light is emitted in a case where use ismade of an electronic flash, shutter speed, EV value, LV value, numberof recorded pixels, pixel mode, reproduction band, f-stop number, colordifference matrix, lens disposition, zoom position, F-value and contourcorrection value, etc.

FIG. 9 is a block diagram illustrating the electrical structure of thenoise reduction circuit 6.

When the CCD-RAW data that has been output from the gamma correctioncircuit 5 enters the noise reduction circuit 6, the data is input to anarray converting circuit 21. The array converting circuit 21 divides theCCD-RAW data, which is output in accordance with the color filter array(see FIG. 1) of CCD 1, on a per-color-component basis in the mannerdescribed above.

The CCD-RAW data that has been divided on a per-color-component basis inthe array converting circuit 21 is input to a filtering-targetextraction circuit 22. The latter extracts CCD-RAW data, whichrepresents pixels in noise-reduction target areas each having fivepixels in both the column and row directions (these areas arerepresented as the noise-reduction target areas AR, AB, AG1 and AG2 inFIG. 2), from the CCD-RAW data that has been divided on theper-color-component basis.

The CCD-RAW data to undergo filtering is input to a filter calculationcircuit 23. The latter calculates a filter F in accordance with Equation(4) below.F={D _((s+n)) −αD _(n) }D _((s+n)) ⁻¹  Equation (4)where D_((s+n)) in Equation (4) is a quantity that contains a signal andnoise and is represented by Equation (5) below. Further, α is a filtercontrol coefficient.

$\begin{matrix}{D_{({s + n})} = \begin{bmatrix}D_{{({s + n})}R} & D_{{({s + n})}{RG1}} & D_{{({s + n})}{RB}} & D_{{({s + n})}{RG2}} \\D_{{({s + n})}{G1R}} & D_{{({s + n})}{G1}} & D_{{({s + n})}{G1B}} & D_{{({s + n})}{G1G2}} \\D_{{({s + n})}{BR}} & D_{{({s + n})}{BG1}} & D_{{({s + n})}B} & D_{{({s + n})}{BG2}} \\D_{{({s + n})}{G2R}} & D_{{({s + n})}{G2G1}} & D_{{({s + n})}{G2B}} & D_{{({s + n})}{G2}}\end{bmatrix}} & {{Equation}\mspace{14mu}(5)}\end{matrix}$

Here a diagonal component of D_((s+n)) is amount of variance of thesignal of each color, and a non-diagonal component is amount of varianceof a signal between colors. D_((s+n)x)(x=R, B, G1, G2) is represented byEquation (6) below, and the non-diagonal component D_((s+n)x1gx2)(x1,x2=R, B, G1, G2) is represented by Equation (7) below.

Further, Dn in Equation (1) is a quantity solely of noise and isrepresented by Equation (6) below.

$\begin{matrix}{D_{n} = \lfloor \begin{matrix}D_{nR} & 0 & 0 & 0 \\0 & D_{nG1} & 0 & 0 \\0 & \; & D_{nB} & 0 \\0 & 0 & 0 & D_{nG2}\end{matrix} \rfloor} & {{Equation}\mspace{14mu}(6)}\end{matrix}$

When the filter F is thus calculated, the calculated filter and theCCD-RAW data representing the pixels in the noise-reduction target areaare input to a filter operation circuit 24. The latter performs a filteroperation (noise reduction processing) that is based on Equation (7)below.

$\begin{matrix}{\lfloor \begin{matrix}R_{out} \\{G1}_{out} \\B_{out} \\{G2}_{out}\end{matrix} \rfloor = {{F\lfloor \begin{matrix}{R_{c} - {avR}} \\{{G1}_{c} - {avG1}} \\{B_{c} - {avB}} \\{{G2}_{c} - {avG2}}\end{matrix} \rfloor} + \lfloor \begin{matrix}{avR} \\{avG1} \\{avB} \\{avG2}\end{matrix} \rfloor}} & {{Equation}\mspace{14mu}(7)}\end{matrix}$

In Equation (7), Rout, G1out, Bout and G2out indicate image data of thered color component, first green color component, blue color componentand second green color component, respectively, obtained following thefilter operation; Rc, G1c, Bc and G2c indicate image data representingnoise-reduction target pixels present at the centers of thenoise-reduction target areas AR, AG1, AB and AG2 of the red colorcomponent, first green color component, blue color component and secondgreen color component, respectively, obtained by color-componentdivision processing; and avR, avG1, avB and avG2 are items of dataindicating average values of image data of pixels in the noise-reductiontarget areas AR, AG1, AB and AG2 of the red color component, first greencolor component, blue color component and second green color component,respectively, obtained by color-component division processing.

In Equation (7), the levels of image data of pixels in thenoise-reduction target areas AR, AG1, AB and AG2 are shifted in themanner described above (see FIG. 5) to the origin position of the colorspace that has the red color component, first green color component,blue color component and second green color component as its coordinatesystem by subtracting the data avR, avG1, avB and avG2 indicating theaverage values of image data of pixels in the noise-reduction targetareas AR, AG1, AB and AG2 of the red color component, first green colorcomponent, blue color component and second green color component,respectively, from the noise-reduction target pixels Rc, G1c, Bc and G2cpresent at the centers of the noise-reduction target areas AR, AG1, ABand AG2 of the red color component, first green color component, bluecolor component and second green color component that have been obtainedby color-component division processing. The thus shifted pixels Rc, G1c,Bc and G2c to undergo noise reduction are subjected to filteringprocessing using the filter F indicated by Equation (4).

When D_(n) indicated by Equation (6) is calculated from D_((s+n))indicated by Equation (5) in the filtering processing using the filter Findicated by Equation (4), the uncorrelated noise of the CCD-RAW data iseliminated because the diagonal component of D_((s+n)) is subtracted,and the correlativity of the CCD-RAW data is maintained because thenon-diagonal component is not subtracted. In other words, uncorrelatednoise is removed while the correlativity of the CCD-RAW data ismaintained.

By adding the data avR, avG1, avB and avG2 indicating the average valuesto the noise-reduction target pixels Rc, G1c, Bc and G2c obtainedfollowing filtering processing, the noise-reduction target pixels Rc,G1c, Bc and G2c are returned to levels corresponding to the originallevels from which noise has been reduced.

In order to prevent parameters from becoming too large and, hence,degradation of the image following noise reduction processing in theabove-described noise reduction processing, it is preferred thatparameters Dgx and Dnx used in Equations (2) and (5) satisfy therelation of Equation (8) below. Here x=R, G1, B, G2 holds.if D _((s+n)) <Dn, then D _((s+n)) =D _(n)  Equation (8)where X=R, G1, B, G2.

Such noise reduction processing is repeated with regard to one frame'sworth of image data. Of course, if the level of color image data (thelevel of a pixel to undergo noise reduction) is equal to or greater thana predetermined level, noise reduction processing may be halted.

The image data that has undergone filter processing in the filteroperation circuit 24 is input to an array reverse-conversion processingcircuit 25. The array of image data that has been divided on aper-color-component basis is returned from the array of the colorfilters of CCD 1 to the original array of color filters of CCD 1 in thearray reverse-conversion processing circuit 25. The output of the arrayreverse-conversion processing circuit 25 is the output of the noisereduction circuit 6.

FIG. 10 is a block diagram illustrating the electrical structure of acomputer system.

The above-described embodiment is such that noise reduction processingis executed in a digital still camera. However, noise reductionprocessing of CCD-RAW data can also be executed utilizing a computersystem.

The computer system includes a CPU 40 to which have been connected adisplay unit 41, a printer 42, a keyboard 43 and a memory 44.

Also connected to the CPU 40 is a memory card reader/writer 45. Byloading a memory card 51 on which CCD-RAW data has been recorded intothe memory card reader/writer 45, the CCD-RAW data is read from thememory card 51 and the data is subjected to the above-described noisereduction processing. A CD-ROM drive 46 is further connected to the CPU40. If a CD-ROM 52 on which a program for the above-described noisereduction processing has been stored is loaded into the CD-ROM drive 46,the noise reduction processing program will be read from the CD-ROM 52and installed in the computer. Noise reduction processing can be appliedto the CCD-RAW data that has been read from the memory card 51.

Further connected to the CPU 40 is a hard-disk drive 47. CCD-RAW datathat has undergone noise reduction processing can be recorded on a harddisk 48 by the hard-disk drive 47.

FIG. 11 is a flowchart illustrating noise reduction processing executedin the computer system.

As mentioned above, CCD-RAW data in which image data of color componentsthat are in accordance with the color-filter array of CCD 1 appears isobtained from the memory card 51. This CCD-RAW data is subjected toarray conversion processing (color-component division processing) insuch a manner that the data will appear collectively on aper-color-component basis in the manner described above (step 31). Whenthe array conversion processing is executed, image data to undergo noisereduction is extracted (step 32).

The filter F is calculated (step 33) and processing is executed usingthe filter F calculated (step 34). The processing from step 32 to step34 is repeated until the filter operation has been applied to the finalpixel of one frame of the image (step 35). This is followed by executingthe above-described processing for reverse conversion of the array (step36).

In the above-described embodiment, the filtering operation in noisereduction processing is performed using color components of four colors.However, a filter operation can also be performed using two colors×twocolor components. For example, an image can be divided into a setcomposed of a red color component and first green color component and aset composed of a blue color component and second green color component.A calculation indicated below can be performed with regard to the setcomposed of the red color component and first green color component. Itgoes without saying that a calculation can be performed in similarfashion with regard to the set composed of the blue color component andsecond green color component.

Equation (5) cited above is represented by Equation (9) below, andEquation (6) cited above is represented by Equation (10) below.

$\begin{matrix}{D_{({s + n})} = \lfloor \begin{matrix}D_{{({s + n})}R} & D_{{({s + n})}{RG1}} \\D_{{({s + n})}{G1R}} & D_{{({s + n})}{G1}}\end{matrix} \rfloor} & {{Equation}\mspace{14mu}(9)} \\{D_{n}\lfloor \begin{matrix}D_{nR} & 0 \\0 & D_{nG1}\end{matrix} \rfloor} & {{Equation}\mspace{14mu}(10)}\end{matrix}$

Equation (7) cited above becomes Equation (11) when Equations (9) and(10) are used.

$\begin{matrix}{\begin{bmatrix}R_{out} \\{G1}_{out}\end{bmatrix} = {{F\begin{bmatrix}{R_{c} - {avR}} \\{{G1}_{c} - {avG1}}\end{bmatrix}} + \begin{bmatrix}{avR} \\{avG1}\end{bmatrix}}} & {{Equation}\mspace{14mu}(11)}\end{matrix}$

Accordingly, in a case where an inverse matrix at the time ofcalculation of F is not found, it is preferred to so arrange it thatnoise reduction processing is not executed.

For example, in a case where a filter operation based upon Equation (4)is performed using Equation (9), D_((s+n)) ⁻¹ is represented by Equation(12) below.

$\begin{matrix}{D_{({s + n})}^{- 1} = {\frac{1}{{AD} - {BC}}\lfloor \begin{matrix}D & {- B} \\{- C} & A\end{matrix} \rfloor}} & {{Equation}\mspace{14mu}(12)}\end{matrix}$where Equation (13) below is written for D_((s+n)) indicated by Equation(9).

$\begin{matrix}{D_{({s + n})} = {\lfloor \begin{matrix}D_{{({s + n})}R} & D_{{({s + n})}{RG1}} \\D_{{({s + n})}{G1R}} & D_{{({s + n})}{G1}}\end{matrix} \rfloor = \begin{bmatrix}A & B \\C & D\end{bmatrix}}} & {{Equation}\mspace{14mu}(13)}\end{matrix}$

Assume that Δ=AD−BC holds in Equation (12). If the value of Δ approacheszero, the image represented by image data that has undergone noisereduction processing will be degraded and therefore noise reductionprocessing is inhibited.

FIGS. 12 to 16 illustrate a modification of the embodiment.

FIG. 12 illustrates an example of the image of a subject represented byCCD-RAW data.

Depending upon the camera angle, there are occasions where an edge 63 isproduced in the image 60 of the subject. There are instances wherecorrelativity vanishes between the level of the image data representingthe image in an image area 61 on the left side of the edge 63 and thelevel of the image data representing the image in an image area 62 onthe right side.

FIG. 13 illustrates a level distribution of image data representingpixels within a noise-reduction target area in the color space of a redcolor component, blue color component, first green color component andsecond green color component.

If the edge 63 is produced in the image 60 of the subject and there isno image correlativity between the areas 61 and 62 on both sides of theedge 63, as shown in FIG. 12, the distribution of image data of pixelsin a noise-reduction target area AE that includes the edge 63 becomesscattered in accordance with the position of a pixel on the image 60 ofthe subject in the noise-reduction target area without falling withinfixed bounds centered on the average value XAV of image datarepresenting pixels in the noise-reduction target area, as mentionedabove. Consequently, even if the average value XAV is shifted to theorigin in color space and filtering processing is executed in the mannerdescribed above, there are instances where the correlativity of pixelsin the noise-reduction target area AE cannot be maintained.

FIG. 14 illustrates an example of a noise-reduction target area AY.

In this modification, CCD-RAW data is converted to luminance data andthe luminance data obtained by the conversion is used.

The noise-reduction target area AY contains a total of 25 pixels 72,namely five pixels in each of the column and row directions. A pixel Ycat the center of these pixels is a pixel to undergo noise reductionprocessing.

In a case where the noise-reduction target area AY involves the edge 63in the manner described above, there are instances where noise reductionprocessing cannot be executed while correlativity between pixels ismaintained. In this modification, therefore, as shown in FIG. 15, anoise-reduction target area AY1 exhibiting correlativity is detectedanew and the above-described noise reduction processing is executedusing the average value of pixels in the detected new noise-reductiontarget area AY1. Noise reduction processing can be executed whilecorrelativity of pixels is maintained.

FIG. 16 is a flowchart illustrating processing for detecting anoise-reduction target area having correlativity.

Luminance data within a noise-reduction target area having five pixelsin each of column and row directions is generated from CCD-RAW data(step 81; see FIG. 14). It is determined whether the difference betweenthe level of image data of one pixel (a pixel to undergo discrimination)within the noise-reduction target area and the level of image data ofthe pixel Yc at the center is less than a predeterminedarea-discrimination threshold value (step 82). If the difference is lessthan the threshold value (“YES” at step 82), then it is construed thatthis one pixel has correlativity with respect to the center pixel Yc.Accordingly, this one pixel, which is the pixel undergoingdiscrimination, is added to a new noise-reduction target area (step 83).If the difference is equal to or greater than the threshold value (“NO”at step 82), then it is construed that this one pixel does not havecorrelativity with respect to the center pixel. This pixel, therefore,is not added to the new noise-reduction target area. The above-describedprocessing of steps 82 and 83 is repeated with regard to all pixels toundergo discrimination (step 84). If the pixel processed is not thefinal pixel (“NO” at step 84), then a pixel neighboring (e.g., on theright side) the one pixel is set as a new pixel to undergodiscrimination (step 85).

If the above-described processing of steps 82 and 83 has been completedfor all pixels (“YES” at step 84), then the new noise-reduction targetarea AY1 is decided as illustrated in FIG. 15.

The method of processing changes further in accordance with the size ofthe new noise-reduction target area AY1 thus decided.

In a case where the size of the new noise-reduction target area AY1 isvery small and is less than a first threshold value (“YES” at step 86),the number of pixels within the noise-reduction target area will be toosmall and there are instances where comparatively appropriate noisereduction processing cannot be executed. As a result, noise reductionprocessing is halted (step 87).

If the size of the noise-reduction target area AY1 is greater than thefirst threshold value (“NO” at step 86) but is comparatively small andtherefore smaller than a second threshold value (first thresholdvalue>second threshold value) (“YES” at step 88), then, in order toenlarge the number of pixels in the noise-reduction target area, noisereduction processing is not executed using the new noise-reductiontarget area AY1 but is executed using a noise-reduction target area of aprescribed size (step 89).

If the size of the new noise-reduction target area AY1 is greater thanthe second threshold value (“NO” at step 88), then it is construed thatthe noise-reduction target area of the prescribed size contains an edgeportion or the like. In order to eliminate the edge portion, noisereduction processing is executed using the new noise-reduction targetarea AY1 (step 90).

The above-described processing for deciding a new noise-reduction targetarea performs discrimination one pixel at a time. However, it may be soarranged that the decision is rendered for every block of a plurality ofpixels.

FIG. 17 illustrates an example of a noise-reduction target area.

As shown in FIG. 17, pixels in a noise-reduction target area are dividedinto individual blocks of every row and column with the exception of therow and column that contain the center pixel Yc. That is, in the rowdirection, the pixels are divided into small blocks P2, P6, P8 and P4 ofa first row, second row, fourth row (the third row is excluded becauseit contains the center pixel Yc) and fifth row, respectively. In thecolumn direction, the pixels are divided into small blocks P3, P7, P5and P1 of a first column, second column, fourth column (the third columnis excluded because it contains the center pixel Yc) and fifth column,respectively.

Among the pixels constituting each small block, the center pixel isadopted as the representative pixel of this small block. Therepresentative pixels of the small blocks P2, P6, P8 and P4 of thefirst, second, fourth and fifth rows, respectively, are Y2, Y6, Y8 andY4, respectively. The representative pixels of the small blocks P3, P7,P5 and P1 of the first, second, fourth and fifth columns, respectively,are Y3, Y7, Y5 and Y1, respectively.

It is determined whether the difference between the level of therepresentative pixel of each block and the level of the center pixel Ycis less than the area-discrimination threshold value, as mentionedabove. If the difference is less than the threshold value, then theentirety of the small block that contains this representative pixel isadded to the new noise-reduction target area.

FIG. 18 illustrates an example of a noise-reduction target area decidedanew.

Pixels of the first row and first column have been excluded from anoise-reduction target area AY2 decided anew. The reason for this isthat with regard to the representative pixels Y2 and Y3, the leveldifference with respect to the center pixel Yc has been determined to beequal to or greater than the predetermined area-discrimination thresholdvalue. With regard to the representative pixels Y6, Y8, Y4, Y7, Y5 andY1 of the small blocks of the other rows and columns, the leveldifference relative to the center pixel is less than the predeterminedarea-discrimination value and therefore the pixels contained in each ofthese small blocks are contained in the noise-reduction target area AY2.

Thus, processing is simplified by discriminating a noise-reductiontarget area on a per-small-block basis.

It goes without saying that the embodiment of the present invention isnot limited to the above-described filter array and can be applied toany filter array. For example, as shown in FIG. 19, the embodiment isalso applicable to a so-called honeycomb array in which a (4n+1)thcolumn, (4n+2)th column, (4n+3)th column and (4n+4)th column areprovided in odd-numbered rows with filters having characteristics thatpass the red color component, first green color component, blue colorcomponent and second green color component, respectively, and the(4n+1)th column, (4n+2)th column, (4n+3)th column and (4n+4)th columnare provided in even-numbered rows with filters having characteristicsthat pass the blue color component, second green color component, redcolor component and first green color component, respectively. Further,as shown in FIG. 20, the embodiment is also applicable to a so-calledBayer array in which odd-numbered rows and odd-numbered columns areprovided with filters having a characteristic that passes the red colorcomponent, odd-numbered rows and even-numbered columns are provided withfilters having a characteristic that passes the first green colorcomponent, even-numbered rows and odd-numbered columns are provided withfilters having a characteristic that passes the second green colorcomponent, and even-numbered rows and even-numbered columns are providedwith filters having a characteristic that passes the blue colorcomponent. Thus, this embodiment is applicable if the array of colorfilters is systematic.

As many apparently widely different embodiments of the present inventioncan be made without departing from the spirit and scope thereof, it isto be understood that the invention is not limited to the specificembodiments thereof except as defined in the appended claims.

1. A noise reduction apparatus comprising: a color-component extractiondevice for inputting image data representing an image in which eachpixel of a number of pixels constituting one frame of an image has onecolor component from among a plurality of color components, the colorcomponents having a distribution that is systematic, and extracting, ona per-color-component basis, color image data representing a pixelwithin a zone in which the color image data is regarded as havingcorrelativity; a noise removal device for removing uncorrelated noise,which is contained in the color image data that has been extracted on aper-color-component basis, while maintaining correlativity of the colorimage data that has been extracted by said color-component extractiondevice on a per-color-component basis; and a control device forexercising control in such a manner that color-component extractionprocessing by said color-component extraction device and noise removalprocessing by said noise removal device is repeated with regard to theone frame of color image data.
 2. The apparatus according to claim 1,further comprising a noise removal processing inhibiting device forinhibiting noise removal processing by said noise removal device inaccordance with level of the color image data.
 3. The apparatusaccording to claim 1, wherein said color-component extraction deviceinputs color image data that is output from a single-chip solid-stateelectronic image sensing device in which a number of optoelectronictransducers are arrayed, color filters each having a characteristic thatpasses light of one color component from among the plurality of colorcomponents being formed systematically, for each of the plurality ofcolors, on photoreceptor surfaces of respective ones of theoptoelectronic transducers of said number of optoelectronic transducers.4. The apparatus according to claim 3, wherein processing for removinguncorrelated noise in said noise removal device is executed based uponthe characteristic of the solid-state electronic image sensing deviceand at least one item of shooting information used when a picture hasbeen taken using said solid-state electronic image sensing device. 5.The apparatus according to claim 1, wherein said noise removal deviceincludes: a color image data shifting device for shifting the level ofthe color image data, which has been extracted by said color-componentextraction device, in such a manner that an average value of the levelsof the color image data that has been extracted by said color-componentextraction device will become an origin position of color space of theplurality of color components; a filtering device for subjecting thecolor image data that has been shifted by said color image data shiftingdevice to processing for removing uncorrelated noise in accordance withthe level of this color image data; and a color image data reverseshifting device for returning, in accordance with the amount of shift,the level of the color image data from which the uncorrelated noise hasbeen removed by said filtering device.
 6. The apparatus according toclaim 5, wherein the noise reduction processing in said filtering deviceis digital filtering processing that utilizes calculation conforming tothe number of the plurality of color components.
 7. The apparatusaccording to claim 5, wherein the noise reduction processing in saidfiltering device is digital filtering processing for handling theplurality of color components in such manner that the number of theplurality of color components will become less than the number oforiginal color components, this processing utilizing calculationconforming to the number of the plurality of color components.
 8. Theapparatus according to claim 7, further comprising a noise reductionprocessing halting device for halting noise reduction processing in acase where calculation in the digital filtering processing has diverged.9. The apparatus according to claim 5, wherein digital filteringprocessing applied to the shifted color image data in said filteringdevice uses processing that subtracts uncorrelated noise from image datain an area that contains uncorrelated noise.
 10. The apparatus accordingto claim 5, further comprising an image data dividing device fordividing the entirety of the color image data that has been extracted bysaid color-component extraction device into a plurality of color imagedata groups exhibiting correlativity with one another; wherein saidcolor image data shifting device performs the shift with regard to acolor image data group, which contains color image data that is toundergo removal of uncorrelated noise, from among the plurality of colorimage data groups obtained by division by said image data dividingdevice, in such a manner that this color image data group will occupythe origin position of the color space of the plurality of colorcomponents.
 11. The apparatus according to claim 10, wherein said imagedata dividing device includes a small-block dividing device for dividingcolor image data that has been extracted by said color-componentextraction device into a plurality of small blocks; wherein the colorimage data is divided into a plurality of correlated color image datagroups per each of the plurality of small blocks obtained by division bysaid small-block dividing device.
 12. The apparatus according to claim11, wherein the color image data is divided into a plurality ofcorrelated color image data groups per each of the plurality of smallblocks based upon representative pixels that constitute the smallblocks.
 13. The apparatus according to claim 10, further comprising anuncorrelated noise halting control device for halting processing forremoving uncorrelated noise by said noise removing device in a casewhere the size of an image area represented by the plurality of colorimage data groups obtained by division by said image data dividingdevice is less than a first predetermined value.
 14. The apparatusaccording to claim 10, wherein in a case where the size of an image arearepresented by the plurality of color image data groups is equal to orgreater than a second predetermined value larger the first predeterminedvalue, the shift is performed with regard to a color image data group,which contains color image data that is to undergo removal ofuncorrelated noise, from among the plurality of color image data groupsobtained by division by said image data dividing device.
 15. Theapparatus according to claim 10, further comprising a luminance datagenerating device for generating luminance data from the color imagedata that has been extracted by said color-component extraction device;wherein said image data dividing device divides the color image datainto a plurality of luminance data groups, which exhibit correlativitywith one another, from among the luminance data that has been generatedby said luminance data generating device.
 16. A method of reducingnoise, comprising the steps of: inputting image data representing animage in which each pixel of a number of pixels constituting one frameof an image has one color component from among a plurality of colorcomponents, the color components having a distribution that issystematic, and extracting, on a per-color-component basis, color imagedata representing a pixel within a zone regarded as havingcorrelativity; removing uncorrelated noise, which is contained in thecolor image data that has been extracted on a per-color-component basis,while maintaining correlativity of the color image data that has beenextracted on a per-color-component basis; and repeating thecolor-component extraction processing and noise removal processing withregard to the one frame of color image data.
 17. A computer-readablemedium containing a noise reduction program for controlling a computerso as to implement the following steps: inputting image datarepresenting an image in which each pixel of a number of pixelsconstituting one frame of an image has one color component from among aplurality of color components, the color components having adistribution that is systematic, and extracting, on aper-color-component basis, color image data representing a pixel withina zone regarded as having correlativity; removing uncorrelated noise,which is contained in the color image data that has been extracted on aper-color-component basis, while maintaining correlativity of the colorimage data that has been extracted on a per-color-component basis; andrepeating the color-component extraction processing and noise removalprocessing with regard to the one frame of color image data.